High-Performance I/O: HDF5 for Lattice QCD

摘要

Practitioners of lattice QCD/QFT have been some of the primary pioneer usersof the state-of-the-art high-performance-computing systems, and contributetowards the stress tests of such new machines as soon as they become available.As with all aspects of high-performance-computing, I/O is becoming anincreasingly specialized component of these systems. In order to take advantageof the latest available high-performance I/O infrastructure, to ensurereliability and backwards compatibility of data files, and to help unify thedata structures used in lattice codes, we have incorporated parallel HDF5 I/Ointo the SciDAC supported USQCD software stack. Here we present the design andimplementation of this I/O framework. Our HDF5 implementation outperformsoptimized QIO at the 10-20% level and leaves room for further improvement byutilizing appropriate dataset chunking.